[MRG] 32/64-bit float consistency with BernoulliRBM #16352

Henley13 · 2020-01-31T17:18:40Z

Reference Issues/PRs

Works on #11000 for BernoulliRBM.

What does this implement/fix? Explain your changes.

Prevent the transformer from converting float32 to float64.

Any other comments?

Maybe we should wait to merge a generic test for dtype consistency (see #16290) before merging this one.

rth · 2020-02-01T14:02:18Z

sklearn/neural_network/tests/test_rbm.py

+
+    # dtype_in and dtype_out consistent
+    assert Xt.dtype == dtype_out, ('transform dtype: {} - original dtype: {}'
+                                   .format(Xt.dtype, X.dtype))


While #16290 is not merged, we also should add a check that the results of fit_transform are close enough with float32 and float64 input.

We also should check that the attributes of the estimator are close on float32 and float64 inputs, which I think is better to be done in the individual tests.

Just added a dedicated test for that.

jnothman

Is this more susceptible to loss of precision in intermediate matrix multiplications?

rth · 2020-02-03T09:13:26Z

Is this more susceptible to loss of precision in intermediate matrix multiplications?

I guess this question could apply to any neural net architecture, and yet all DL libraries use float32 and sometimes even floa16 lately. Maybe they just don't enforce that outputs in f64/f32 are identical? Not sure. If so, I wonder whether we should. Possibly checking that the convergence critera was reached no matter what it was is enough (at least for some type of algorithms).

jeremiedbb

lgtm. Just a small comment. Also please add a what's new entry

jeremiedbb · 2020-02-12T12:38:54Z

sklearn/neural_network/tests/test_rbm.py

+    assert_almost_equal(Xt_64, Xt_32, 6)
+    assert_almost_equal(rbm_64.intercept_hidden_,
+                        rbm_32.intercept_hidden_,
+                        6)
+    assert_almost_equal(rbm_64.intercept_visible_,
+                        rbm_32.intercept_visible_,
+                        6)
+    assert_almost_equal(rbm_64.components_, rbm_32.components_, 6)
+    assert_almost_equal(rbm_64.h_samples_, rbm_32.h_samples_, 0)


Please use assert_allclose instead

jeremiedbb · 2020-02-12T12:40:32Z

sklearn/neural_network/_rbm.py

@@ -146,8 +146,10 @@ def _mean_hiddens(self, v):
        h : ndarray of shape (n_samples, n_components)
            Corresponding mean field values for the hidden layer.
        """
+


Suggested change

jeremiedbb · 2020-02-12T12:40:40Z

sklearn/neural_network/_rbm.py

        p = safe_sparse_dot(v, self.components_.T)
        p += self.intercept_hidden_
+


Suggested change

…ing_dtypes

rth

LGTM, thanks!

rth · 2020-02-12T22:37:11Z

doc/whats_new/v0.23.rst

@@ -235,6 +235,10 @@ Changelog
  :class:`neural_network.MLPClassifier` by clipping the probabilities.
  :pr:`16117` by `Thomas Fan`_.

+- |Enhancement| Prevent the transformer from converting float32 to float64 in


Maybe "Avoid converting float32 input to float64 in ..."?

Much better ^^

…ing_dtypes

rth · 2020-06-23T22:32:16Z

Thanks @Henley13 !

Henley13 added 3 commits January 31, 2020 11:10

add a quick test

f24b9e9

first solution

ac18ba7

improve test

fee05d5

Henley13 changed the title ~~[MRG] Bernoulli preserving dtypes~~ [MRG] 32/64-bit float consistency with BernoulliRBM Jan 31, 2020

agramfort approved these changes Jan 31, 2020

View reviewed changes

Henley13 added 2 commits January 31, 2020 18:34

fix error by setting random_state manually

31fdb98

rename properly test

ad385ad

rth reviewed Feb 1, 2020

View reviewed changes

jnothman reviewed Feb 3, 2020

View reviewed changes

rth added the Needs work label Feb 5, 2020

Henley13 added 2 commits February 11, 2020 16:52

update branch from upstream/master

47b7dec

add test to compare 32-64 bit results

c93b2ae

jeremiedbb approved these changes Feb 12, 2020

View reviewed changes

Henley13 added 3 commits February 12, 2020 17:09

Merge remote-tracking branch 'upstream/master' into bernoulli_preserv…

4e768a2

…ing_dtypes

use 'assert_allclose' in the test

d2259ce

update whats_new

dfb51bc

rth approved these changes Feb 12, 2020

View reviewed changes

Henley13 added 2 commits February 13, 2020 10:45

update doc whats_new

8dc88d9

Merge remote-tracking branch 'upstream/master' into bernoulli_preserv…

dc4660c

…ing_dtypes

github-actions bot added the module:neural_network label Mar 2, 2020

rth self-requested a review June 22, 2020 12:54

rth added 2 commits June 23, 2020 21:15

Merge remote-tracking branch 'upstream/master' into bernoulli_preserv…

0133591

…ing_dtypes

Merge branch 'master' into bernoulli_preserving_dtypes

4facc85

rth removed the Needs work label Jun 23, 2020

rth merged commit 3583149 into scikit-learn:master Jun 23, 2020

rth mentioned this pull request Jun 24, 2020

Add 32 bit support to neural_network module #17700

Closed

rubywerman pushed a commit to MLH-Fellowship/scikit-learn that referenced this pull request Jun 24, 2020

ENH 32/64-bit float consistency with BernoulliRBM (scikit-learn#16352)

44e51ad

jayzed82 pushed a commit to jayzed82/scikit-learn that referenced this pull request Oct 22, 2020

ENH 32/64-bit float consistency with BernoulliRBM (scikit-learn#16352)

d1b9f46

jayzed82 pushed a commit to jayzed82/scikit-learn that referenced this pull request Oct 22, 2020

ENH 32/64-bit float consistency with BernoulliRBM (scikit-learn#16352)

b087011

This was referenced Sep 2, 2022

Preserving dtype for float32 / float64 in transformers #11000

Open

[ENH] add dtype preservation to BernoulliRBM #24318

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[MRG] 32/64-bit float consistency with BernoulliRBM #16352

[MRG] 32/64-bit float consistency with BernoulliRBM #16352

Henley13 commented Jan 31, 2020

rth Feb 1, 2020

ksslng Feb 2, 2020 •

edited

Henley13 Feb 11, 2020

jnothman left a comment

rth commented Feb 3, 2020 •

edited

jeremiedbb left a comment •

edited

jeremiedbb Feb 12, 2020

Henley13 Feb 12, 2020

jeremiedbb Feb 12, 2020

jeremiedbb Feb 12, 2020

rth left a comment

rth Feb 12, 2020

Henley13 Feb 13, 2020

rth commented Jun 23, 2020

		p = safe_sparse_dot(v, self.components_.T)
		p += self.intercept_hidden_

[MRG] 32/64-bit float consistency with BernoulliRBM #16352

[MRG] 32/64-bit float consistency with BernoulliRBM #16352

Conversation

Henley13 commented Jan 31, 2020

Reference Issues/PRs

What does this implement/fix? Explain your changes.

Any other comments?

Choose a reason for hiding this comment

ksslng Feb 2, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jnothman left a comment

Choose a reason for hiding this comment

rth commented Feb 3, 2020 • edited

jeremiedbb left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rth commented Jun 23, 2020

ksslng Feb 2, 2020 •

edited

rth commented Feb 3, 2020 •

edited

jeremiedbb left a comment •

edited